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ENHANCED CUSTOM CONTENT TELEVISION 



BACKGROUND 

Historically, television content has been based on a business model of a 
common network programming media content being pushed to a mass nationwide 
audience. Local network television broadcasters had the option of inserting local 
advertising in some of the commercial breaks or perhaps pre-empting the network 
programming for local programming. Still, the effect was the same of pushing a 
common content on a mass audience, thought limited in a geographic region. 

Past efforts in "interactive T.V." have focused on a variety of "interactions" 
including an interactive television programming guide, video-on-demand, pay-per- 
view, and Internet web pages displayed on the television. These latter efforts have 
maintained a clear difference between the web pages and television programming. 
The Internet content typically appears in a separate PIP-format window overlaying 
or next to the television program. 

What has been lacking, however, has been a means to seamlessly merge the 
two content sources. What is needed is a richer television viewing experience that 
provides customized, personalized, interactive enhancements that can take 
advantage of today's and tomorrow's higher broadband connectivity with 
alternative multimedia content sources and low-cost, powerful graphics and multi- 
media image processing technology. 

SUMMARY 

The present invention is intended to take advantage of today's increased 
multimedia processing capability to provide customized personalized interactive 



enhancements to television broadcast programs. In a first embodiment, the present 
invention is directed to a method for enhancing a television broadcast program by 
receiving television broadcast programs and broadcast advertising segments, 
receiving replacement advertising data where the advertising data represents an 
instruction set for rendering into an animated video segment by a client processor, 
rendering on the client processor the replacement advertising segment, blocking 
the display of the television broadcast advertising, and displaying the replacement 
advertising segment on a television display. 

In a second embodiment of the present invention, a method is provided for 
enhancing a television broadcast program by receiving programming data 
representing synchronization data for a plurality of sequential program segments 
in a television broadcast program, receiving information related to a plurality of 
replacement program segments, selecting a desired replacement segment, 
synchronizing the replacement segment with one of the plurality of television 
broadcast segments, receiving data representing the selected replacement 
segments, blocking the display of the television broadcast segment, and then 
displaying the replacement segment on a television display in place of the 
synchronized television broadcast segment. Preferably, the replacement segment 
is selected based upon a viewer profile. Optionally, the data representing the 
selected replacement segment may comprise an instruction set for rendering into 
an animated video segment by a client processor. 

In a third embodiment of the present invention, a method is provided for 
enhancing a television broadcast program comprising receiving the television 
broadcast program, receiving supplemental informational data from a first memory 
storage, rendering on a client processor supplemental video content from the 
supplemental data and displaying the television program with the supplemental 
video content images displayed over portions of the television program on a 
television display. Preferably, the supplemental video image content is 
informationally related to the displayed television broadcast program. Preferably, 
the informational data represents financial information associated with the content 
of the television broadcast program or with the view profile. Optionally, the 



supplemental video content images may be altered in response to an input from a 
manual input device under the control of a viewer. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a schematic of one embodiment of a system for providing 
enhanced custom contact television. 

FIGS. 2A, 2B and 2C are representative images from a client-rendered 
advertising segment. 

FIGS. 3A and 3B are representative images from an enhanced financial 
news television program. 

DETAILED DESCRIPTION OF THE INVENTION 

The seamless merger of television programming and additional or 
alternative multimedia content sources is referred to herein as "enhanced 
television." The present invention is directed to a system and method for 
combining television broadcast programming content with customized or 
personalized information and educational or entertainment content in a way that 
the television viewer sees a coherent seamless merging of the content sources in 
an indistinguishable format. Preferably, the viewer may interact in real-time with 
the enhanced television programming to provide a higher level of engagement and 
customization potential. 

Recently, commercially available television set-top boxes and video game 
consoles have both Internet connectivity and impressive graphics performance. 
This year, the introduction of video game machines such as the Sony Play Station 
2 and the Microsoft Xbox means that the typical living room now has more 
multimedia processing power than most personal computers. New television sets 
are widely available that have digital capabilities. Personal digital video 
recorders, such as available from TiVo, Inc., allow for the time shifting of 
television broadcasts at a time sequence and period desired by and under the local 
control of the individual viewer. 



All these technologies provide a robust platform for rich media 
applications. This equipment paves the way to implement and generate 
customized television quality content on individual television sets. Examples may 
include customized advertising, customized news programming, individualized 
commercials and unique connectivity with the on-line community through the 
television sets. Thus, in contrast to present attempts at interactive television that 
concentrate on doing most of the data processing on the backend systems and 
streaming video media out to the viewer, certain embodiments of the present 
invention takes advantage of the client-side technology to generate locally for each 
viewer the displayed video media with a more emersive level of interactivity than 
merely selecting different media streams delivered by a backend server. 

One embodiment of a system for providing enhanced custom content 
television programming is depicted in Figure 1 . The television set-top box 10 is 
the control center for providing the personalized television programming. 
Although referred to herein as a "set-top box," the enhanced television controller 
may be located in a variety of places in a viewer's home, or incorporated with 
other hardware such as, but not limited to, the television video display monitor, a 
separate personal computer, home theater audio/visual equipment, etc. Also, the 
set-top box controller may be located off the premises such as at a local cable 
provider's distribution center, which would require broadband connectivity with 
other components of the system. 

Preferably, the set-top box 10 has a video output connection to a television 
set or video display monitor 12. The video output could be either analog or 
digital, depending on the input capabilities of the monitor 12. The display monitor 
12 may be adapted to include a touch sensitive input device 14 overlaying the 
display screen 16. The set-top box may be operated through use of a remote 
controller 18 using on-screen menus for more complicated commands. 

Alternatively, the remote controller 1 8 may be a personal digital assistant 
(PDA) with a touch sensitive display and an infrared tranceiver for communication 
with the set-top box. The set-top box may transmit a data set for the PDA to use 
to reconfigure the display into a set of buttons that correspond to the active 



selections or menu available for interaction with a given image being displayed on 
the monitor 12. For normal television programming, the active buttons may be 
volume adjustment, channel adjustment, etc. For an interactive enhanced 
programming, the active buttons may include fast, slow, freeze, color adjustment, 
camera angle, zoom-in/out. Additional menu buttons on the PDA display may 
include specific links to display graphs, charts, or links to additional overlay 
content, selection of background images, selection of background music, etc. 
depending on the image displayed on the monitor, and level of detail traveled in 
the hierarchical menu of selections. 

The set-top box 10 receives television broadcast programming signals 20 
either through standard wireless transmissions 22, broadband cable connections, 
broadband satellite transmissions, or other means for connectivity. The set-top 
box is also connected through a broadband connection, such as through the 
internet 24 or other external communications networks, to data sources 26 and 
multimedia content sources 28. Multimedia content providers 34 may provide 
interrelated multimedia content for both television broadcasters 20 and multimedia 
sources 28. This supplemental multimedia content is received by the set-top boxes 
and processed for enhanced custom content television viewing on an individual's 
home television 12. Independent multimedia content providers may also provide 
the enhanced supplemental content to display with the television programming. 
Optionally, the added multimedia content may come from CD-ROM's or DVD's 
placed in the optical drive 30 of the set-top box 12. 

The set-top box may also be connected with an individual's home computer 
30 for access to personal data stored on the individual's personal computer ("PC") 
or is available through the PC's connection with the internet 24 or other external 
communications network. Likewise, the PC may be used as an alternative 
interface for programming or setting up the set-top box for personalized enhanced 
television programming. The PC may also store a music catalog owned by the 
viewer from which the set- top box may retrieve the viewer's favorite music for 
customized audio accompaniments or backgrounds for enhanced programming as 
described in further detail below. 



Preferably, the set-top box 10 has the sufficient image processing, graphics 
capability and memory storage to seamlessly integrate the television broadcast and 
digital multimedia content in real time. The set-top box may accept analog or 
digital television broadcasts, strip off embedded codes in the broadcast that 
instructs the set-top box from where multimedia content sources 28 or additional 
data content 26 can be located and downloaded for integrating with the television 
broadcast. The content sources 28 may provide alternative content options that the 
set-top box may select based upon the individual's preferences or profile. 

The system may customize or personalize the television broadcasts 
automatically based on viewer's stored preferences profiles. The preferences or 
profiles may be stored on the set-top box, on the PC, at the content source, or on 
other storage devices that the set-top box is in communication with either in the 
viewer's home or outside. The preferences and profiles may be stored for a 
number of different individuals, i.e., family members, regularly watching the 
enhanced television content. The system may also have preferences established by 
day and time segments to automatically use preferences for viewing hours 
generally watched by children, teens, young adults or parents in the home. The 
system preferably may use these default preferences, or a viewer may input a code 
representing his identity so the system will deliver television programming 
enhanced for his preferences rather than the default preference. 

A viewer's profile or preference may consist of a variety of data or 
information that characterizes the demographics and/or viewing preferences of the 
viewers in the household. For example, the profile may include demographic 
information such as age, gender, ethnicity, income level, zip code and the like. 
Also, profiles could include information relating to a person's financial 
information such as any investment portfolios and information related to account 
numbers and URLs where such information can be located and found on the 
internet. A profile could also include a list of information of major purchases of 
the viewer for possibly a wish list of future purchases desired by the viewer. 
Preferences may include viewing specifics such as types of programs frequently 
watched, hobbies or other interests. Preferences may also include shopping or 



purchasing habits of the viewer. Preferences could also be related to the level of 
interactivity desired by the viewer with the television enhancements. In many 
cases, data attributable to a preference may also be considered a profile, and 
information that is part of the profile could also be considered to be a viewer 
preference. The description of preferences and profiles and the information that 
would be suitable for one or the other will be apparent in connection with the 
description of the preferred methods of the various embodiments of the invention 
as discussed further below. 

The set-top box 10 preferably can store both the television broadcast signal 
and the supplemental multimedia and data content, and then synchronize and 
assemble the multiple content for display in real-time, or at a later time depending 
on the individuals viewers preferences. Since the system may provide complete 
interactivity, the viewer may be able to adjust the content being viewed "on the 
fly." To accomplish this, the system preferably has the video image processing 
capability to render photo-realistic video images from instructions sets or other 
such information data obtained from the supplemental content data sources. From 
this data, the client processor in the set-top box may render three-dimensional 
video images, much like shared data communicated between clients on a 
networked computer video game system can render multi-player video game 
images. The system may be able to "reconfigure" the merged content nearly 
instantaneously by replacing the supplemental content at command of the viewer. 
The system may preferably be able to freeze the display, replay, slow motion, and 
fast forward the video and audio stream while capturing and storing all of the 
desired incoming broadcasts for viewing seconds, minutes, hours or days later, as 
desired by the individual viewer. 

In a preferred embodiment, the set-top box may be considered to have the 
capabilities of personal digital television recorders, internet-capable video game 
players and computer video workstations. The digitized television signals and 
computer-generated supplemental video images from the set-top box being 
displayed preferably appear in the same quality as television broadcast signals 
today. 



For example, such a system may include a set-top box having an Intel 
Pentium III processor with 256 MB RAM memory, a graphics card having a 
nVidia geForce3 chip, a DVD optical drive, a television (or a SVGA computer 
monitor) and a keyboard in communication with the set-top box. The incoming 
television broadcast program may be digitized and stored as a MPEG4 format file 
on a 40 GB hard drive memory storage. The supplemental video content may be 
an executable instruction set written in Visual C++ and using Microsoft Direct X8 
SKD. The system is capable to transmit to the monitor for display the television 
broadcast program, the rendered supplemental video image content, or a 
combination of the supplemental video on top of and/or synchronized with the 
television broadcast program. 

A system having the aforementioned capabilities may be suitable for 
providing a variety of enhanced television viewing modalities. 

The present invention includes several embodiments of methods that are 
directed at enhancing the advertising experience with television. With a set-top 
box that is capable of rendering photo-realistic video images in real time from 
downloaded instruction sets, akin to high resolution computer 3-D video games, a 
viewer can experience the quality of normal television broadcast advertising in a 
creative and engaging experience. 

In one such embodiment, the method provides an enhanced television 
broadcast program with replacement advertisements rendered on the set-top box 
and displayed instead of the television broadcast advertisement accompanying the 
program. Preferably, the replacement advertising segment is selected based on a 
viewer profile or preference stored in the system. The replacement advertising 
segment may be altered or modified by the viewer interacting with the segment 
and selecting a variety of commands. 

For example, during an automobile commercial, as depicted in FIGS. 2 A, B 
and C, the viewer can interact with the commercial to change vehicle options or 
colors of the displayed vehicle to suit his taste. With the viewer's set-top box 
rendering the animated video images of the car commercial, the vehicle 40 can be 
changed to be viewed from any preferred perspective as shown in FIGS. 2 A, B or 



C. Likewise, options to alter the appearance of the vehicle may be selected 
through the selection of menu buttons 42 shown on the left of the display. These 
options may include color changes 44, seat configuration 46, sunroof 48, roof rack 
50, tire style 52, save selection 54 and exit advertisement 56. With computer- 
generated images, special effects, such as ripple-wave color transitions (FIG. 2A) 
or multi-zone ripple-color transitions to appear as a color explosion (FIG. 2B) are 
at the command of the viewer. Typically, special effects are computer-generated 
images that are unrealistic, i.e., do not normally occur in the known environment. 
But the special effects in this embodiment may also include changes in the 
background, such as switching the commercial's background color, or changing 
the location scenery in the background, e.g., country-side to urban environments. 
Likewise, interior configuration changes may be visible by changing the vehicle 
exterior to a "transparent skin" (FIG. 2C). 

The soundtrack accompanying the commercial can be altered depending on 
the taste of the individual viewer. This video and audio content can be delivered 
by the multimedia content source and selected based on demographics or profiles 
of the viewer, or selected form the viewer's favorite music stored locally on the 
set-top box or PC. Likewise, the background scenery may change depending on 
the viewer profiles. For example, the background may be mountain scenery for 
nature lovers, or city background for urban dwellers. 

The viewer may interact with the commercial to stop or freeze the video, 
go backwards or forward on a commercial and spend as much time as he would 
like with the particular product, rather than the present 15 or 30 seconds of a fixed 
commercial. Since the set-top box preferably has the ability to record television 
while displaying personalized multimedia programming, after the viewer has his 
"feel" of the commercial personalized to his preferences, the set-top box can store 
those preferences in memory and then begin displaying the delayed broadcast 
program. 

In addition, the set top box can retain the particular settings that a viewer 
may have selected for a commercial as part of the viewer preferences so that future 
commercials of that product will be display with the same settings. Likewise, the 



-10- 



set top box can "learn" of the viewer's preferences by his interaction with the 
commercial, and thereby build a viewer profile that may be used to automatically 
select certain actions on other commercials that meet the interest captured in the 
viewer's profile. This means that advertisers can deliver commercial content that 
includes optional substitutable content that can be customized or personalized 
based on viewer's profiles or demographics. 

In addition to customizing individual commercials, the viewer may have 
the option of seeing alternative commercials. During a commercial break for a 
television program a variety of different commercials of different products may be 
selectable by the viewer such as selecting between commercials for insurance or 
commercials for a pizza. Alternatively, if a company decides not to share its time 
with other companies (and preferably pays a premium to the broadcaster for that 
privilege), it can offer a selection among a variety of its own products to viewers 
during that time slot based on the viewers' profiles. One example of this may be 
illustrated with an automobile commercial. A recent purchaser of an automobile 
may receive a commercial for auto service, whereas a person who has not 
purchased an automobile recently would receive a commercial for a new 
automobile. 

The custom commercials may also be enhanced by allowing interactivity 
and connectivity to the advertiser so that the viewer can place orders 
instantaneously upon viewing the commercial. Thus, delivering content of more 
interest, intimacy and immediacy to the viewer can strengthen the relationship 
between the viewer and the seller. This can also be used to capture more sellers 
by providing viewers the means to make impulse purchases while viewing a 
commercial. 

The present invention, in other embodiments, can be used to achieve a goal 
long sought by television advertisers, that is, to engage and absorb the attention of 
the television viewing audience. Viewers attention may enhanced by turning 
television commercials into interactive video games. For example, a television 
commercial may include game contests requiring the viewer to view the 
commercial over and over again to discern level of details to answer questions 



about the commercial for prizes. In contrast to video streams, executable code, 
geometry and textures can be quickly transmitted to the viewer's set-top box. As a 
three-dimensional graphic animation being rendered on the set-top box, the 
commercial may be displayed from a variety of different perspectives. In the 
game contest, certain events or information or objects may be seen only at certain 
angles or perspectives are necessary for the television viewer to view to obtain 
appropriate information to answer the quiz or contests. 

The commercials may use the enhanced interactivity to set up 
"advergaming" experiences to draw the viewer in a more intimate relationship 
with the advertised product. One example may be an interactive beer commercial 
where the viewer directs the action or interacts with talking frogs. Prizes 
including merchandise displaying the particular brand of beer logo may be offered 
to motivate the viewer to try to achieve a certain goal within a short time 
appropriate to a commercial, e.g., 30 seconds, such as getting the frogs to say 
certain things, or jump to a certain lillypad, for example. Such "advergaming" 
may be applied on a networked community level so that the action of the directed 
objects are dictated by the popular vote of the community interacting with the 
commercial in real-time. Such a community-directed action may build a "pseudo- 
community" among the viewers that can further enhance the product's image by 
the shared collective experience of consumers interacting with their peers both 
during the commercial and later talking about the "shared experience" when 
communicating with each other. The games or contest may have direct relation to 
the marketed product or be unrelated to the product, using the product as a 
commercial lead-in for the interactive gaming segment "hosted" by the marketed 
product. 

These interactive games in place of traditional commercials may be 
extended to longer programming. For example, children's television cartoons, 
some of which today are computer generated animation, may be converted to 
interactive video games, or include optional segments that are interactive video 
game-type scenes. The children may direct the plot development, or control the 
characters to help the heroes fight the evil villains, or vice versa. 
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In addition to commercials that are purely computer-generated images, the 
commercials can be combinations of real images with computer-generated virtual 
objects superimposed on the real image. While this result has already been seen in 
movies, it can also be accomplished in accordance with another embodiment of 
this invention. For example, it is very common to see product placements in 
movies and T.V. shows that are paid for by the seller of the product. This idea can 
be further augmented so that the product placements on television programs are 
virtual and sold per display by generating the virtual visible products 
superimposed on the real television program through computer-generated 
animation by the set-top box at the viewer's home. Thus, every time a show is 
broadcast or re-run, the television broadcasters have an opportunity to re-sell the 
soda can product placement to a different vendor. Also, a selection of a variety of 
product placements can be provided that are customized to the preferences, 
profiles, or demographics of the individual viewer. 

The present invention may also provide a method for enhancing the content 
of other advertising concepts on television. Applying the interactive mode for 
allowing the viewer to interact and direct the content and progression of 
advertising commercials, the viewer can obtain more tailored content for his 
personal needs. For example, commercials for financial products, such as 
mortgages, insurance, or other investments may allow the viewer to select 
particular products and have visualizations or graphs displayed showing the 
relative investment returns, net present values, income, cash flow or payments 
streams associated with those products. Having the viewer selecting a button on 
the television screen may send a signal via the internet to a content server that will 
transmit executable programming code to the viewer's set top box with 
instructions to render the particular visualizations selected by the viewer. 

This method may be adapted and applied to enhance a variety of additional 
television commercial genres. Retail product commercials may include interactive 
visual product comparisons (such as in the form of graphs, bar charts, etc.) or 
additional product details to provide a virtual comparison shopping experience. 
Displayed buttons linking to information content from independent product 
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shopping service internet servers may provide a wealth of relevant information for 
the viewer, and can be tailored based on the viewers stored profile or preferences. 
For example, a commercial for a new drug may at the option of the viewer provide 
additional graphic comparisons of effectiveness with drugs that the viewer may 
currently be consuming. Graphic animations of how the drugs work may be 
displayed. Links may be provided for additional information related to the 
diseases that the drugs are intended to treat that may be displayed superimposed 
on the commercial, or as a replacement to the commercial display. 

Commercials related to food products may also be enhanced with the 
present invention. For example, a commercial for a dry-packaged rice product 
may be displayed and either at the real-time command of the viewer, or based on 
the viewers profiles and preferences, additional content related to alternative 
recipes to the viewer's tastes may be displayed. An animated interactive 
simulation may be displayed allowing the viewer to virtually trying cooking the 
perfect rice dish, as might be envisioned in a realistic video computer game. 
Another example in the food products category may be a customized pizza 
commercial that displays only the types of pizzas with the ingredients preferred by 
the viewer. Likewise, television commercials for restaurants may be tailored to 
provide commercials for the types of food or ethnic dining preferred by the 
viewer's shopping or buying habits or as previously set-up by the viewer in his 
preferences. 

Likewise, for the potential tourist, vacation commercials may be enhanced 
with selectable content that provides virtual tours or scenic 360-degree panoramas 
of tourist destination sites to motivate the viewer to select a vacation package or 
destination. Airlines may offer similar commercials with interactive options 
allowing viewers to see new features of the airlines, or even reserving flights or 
selecting seats for booked tickets. These commercials may be presented to the 
viewer based on past travel experiences or preferences stored in the system. Hotel 
guests may have locally generated interactive commercials displayed on the hotel 
room televisions showing local tourist attractions, hotel services, or locally 
provided entertainment experiences, such as casino/gambling games or contests. 
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Entertainment media may also take advantage of the present invention. 
New movies and new music albums may present trailers, samples, clips or sound 
bytes customized based on the viewer's profile or selected in real-time by the 
viewer. These short segments may selectively replace the television broadcast 
commercials that would be of no interest to the viewer based on their profiles. 

Because the present invention, in one aspect, provides client-rendered 
video animation superimposed on television broadcast programs, a method is 
provided to display a television broadcast program with superimposed client- 
rendered video images. These images are preferably informationally related to the 
broadcast program and selected based on the viewer profiles. For example, one 
embodiment is an enhanced television newscast based on the concept of a 
financial news television programming as shown in FIGS. 3 A and 3B. In a typical 
financial news broadcast 60, reporters, analysts and experts discuss the latest news 
surrounding specific businesses or markets. In a preferred embodiment, the 
system can be programmed to display a variety of additional information content 
to enhance the information provided by the broadcast. Instead of the common 
variety of stock ticker quotes shown across the bottom of the screen that are under 
the control of the broadcaster or stock exchange, the individual viewer can select 
to have stock quotes or index quotes of his own choosing running across the 
bottom ticker tape 62 of the television screen. 

The enhanced financial news broadcast may be enhanced in other ways. At 
the viewer's request, a chart 64 may be displayed on a television screen showing 
the latest stock price movement of the business that the news reporters are 
currently discussing or other stocks of the viewer's choosing. Also, the viewer 
may switch the views and formats of the information that appears on the screen at 
his choosing using menu buttons 66 on the screen. Likewise, the viewer can call 
up information regarding a personal portfolio of stocks. He may have the ability 
to annotate the system to record 68 the stock currently being discussed by the 
television programming and have the system and the intelligent internet agents 
gather information for research concerning a stock to be viewed at a later point 
and time. In addition, the information and programming can be connected with an 
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online stockbroker so that the viewer can instantly carry out stock trades while 
viewing the customized programming. 

This embodiment of a financial news broadcast and other embodiments 
may be implemented in different ways. For example, the data layer on the 
television broadcast signal may include encoded information in the vertical 
blanking interval (VBI) that tells the set top box to retrieve information from a 
specific Internet site or from an internal storage device such as a personal 
computer that includes information about the viewer's portfolio. Alternatively, the 
electronic programming guide from the cable provider may include such 
information synchronized to the specific relevant portion of the program. 

The viewer can interact with the set-top box by using a remote control, or a 
keyboard connected to the top box or personal computer. Also, the viewer may 
use a touch-sensitive screen over the video display to take advantage of hidden 
and/or visible buttons being displayed along with the television programming. In 
addition, on-screen menus, such as drop-down or pull-down menus, may be used. 

Displaying stock quotes and financial charts may not require as much 
multimedia graphics processing power or capabilities. But, other embodiments of 
the present invention that involve seamless integration of realistic animation with 
television programming may require such advance processing capabilities. The 
set-top box and related equipment preferably has the storage capacity, image 
processing power and broadband connectivity to be able to download and display 
video images with a high degree of three-dimensional realism in real time. 
Moreover the equipment needs to have the image processing capability or graphics 
processing capability to be able to animate three-dimensional images in real time. 

In another embodiment of the invention, the viewer may take control of a 
live television broadcast much as a television program director will do in editing a 
television broadcast in real time. For example, most sporting events are televised 
using cameras located throughout the sports arena. The present invention may 
give the viewer the power and control to select from which camera he desires to 
view the event. The viewer may have control to do instant replays, slow motion, 
and add sports statistical data overlays on the screen of the league, the team, or 
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individual players. Also, the scores of other games in progress may be shown in 
response to the command of the viewer. Computer generated images may be 
displayed, as is currently provided for football game broadcasts such as first-down 
line, player movement chalk-talks, billboard advertisements, etc. However, these 
enhancements may be provided at the command of the individual viewer, or by 
default based on the individual viewer's preferences. Likewise, the viewer may 
select alternate sportscasters or commentators to see and listen to. Some of this 
alternate content may be received from multimedia content providers delivering 
the television broadcast or from independent providers. 

In another embodiment of the invention, instead of augmenting the content 
by overlaying different sources of data information on the television broadcast, the 
viewer may customize news broadcast by requesting a personalized sequence of 
news segments. In this embodiment, a half-hour television news program can be 
prepared and customized for many viewers by taping multiple news segments that 
may total one hour, and delivering to the individual television viewer only those 
segments that he desires or has an interest in viewing. For example, five minutes 
of a common news broadcast to all viewers can be followed by another five 
minutes of news segments where viewers have options of selecting between three 
or four different topical news segments, such as movie reviews, professional 
sports, local high school sports, or international sports. This may be followed by 
again another segment of comment news that is broadcast to everyone followed 
again by customized new segments that are either selected by the viewer or 
perhaps match the television viewers personal profile that is kept on a memory 
storage device. The memory storage device could be the set top box which then 
access the controller of the information and sends out a request for the specific 
segment that match the television viewer's criteria for streaming either in real time 
or ahead of time to the setup box for viewing at the appropriate time and sequence 
of the new broadcast. 

The television broadcasting company may transmit all programs 
simultaneously and the set top box selects from which channel to grab the new 
segment to be displayed during the desired time segment. Alternatively, at the 
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beginning of the program, the television broadcast may send a series of signals and 
with the time sequence and synchronization information with a list of URLs from 
which the set top box can retrieve the stored video new segments for video 
streaming or downloading and later display. Alternatively, the synchronization 
5 information and URLs may be transmitted with electronic programming guides 

and decoded by the set-top box. 

The above various embodiments of the invention may be applied to other 
television programming in the news genre. Sports news programming may be 
enhanced with specific sports or athlete profiles. Plays of the week, month, year 

ffl> or century may be relived with video animation re-enactments where the viewer 

can control the viewing angle, replay speed and other features. Entertainment 

\- news programs may be similarly enhanced with celebrity profiles. Political news 

programs may be enhanced with profiles or segments about a viewer's political 

=v] representatives or political leaders in the current news. Interactive polls may be 

15 taken and/or the results graphically displayed. 

Much like the financial news program enhancements described above, 

=0 weather news channels may be enhanced. Weather news of interest to the viewer 

■.k may be instantly displayed, such as local weather, or upcoming travel destinations. 

The weather channel may include interactive buttons to allow the viewer to 

20 display client-generated graphics of weather information, such as temperature 

forecasts, either in a time graph or an geographic iso-temp map. Graphic 
animations showing satellite and radar information may be displayed on request of 
the user, for example. Realistic video animations of a weather news broadcaster 
may be rendered by the set- top box showing the animated weather person standing 

25 in front of a weather map providing the weather information of interest. 

Of course, one skilled in the art may apply the invention to other 
programming genres based on the teaching herein. For example, a music video 
program may include a variety of different music video segments depending on 
the viewer's preferences. The background set on which the video disk jockey 

30 operates may be generated by the set-top box to provide a background that the 

viewer find most pleasing, at the viewer's command or generated based on the 



-18- 



viewer's profiles. As well, celebrity profiles of interest may be selected during or 

instead of music videos. 

Further, home shopping television programs and the like may be enhanced 

using the present invention. Product comparisons may be displayed. The viewer 
5 may select products of interest for comparison. Additional product details of 

interest to the viewer may be displayed. The products may be shown in the 

viewer's favorite colors or styles, etc. much as described above for the automobile 

commercial embodiment. 

Educational programs may also be enhanced using various embodiments of 
10 the present invention. For example, nature or wildlife programs may include 

=0 interactive set-top box animated features that allow the viewer to feed a wild 

animal, such as a shark, while a narrator talks about the animal's feeding habits or 

other related educational information. The viewer may select to have graphic 
±i visualizations of animal comparisons superimposed on the normal television 

J5 program being displayed, such as showing relative size, speed, food consumption, 

% heart rate, life span, or other pertinent characteristics of the wild animals alone or 

J in comparison with other species. 

■r= While this invention has been shown and described in connection with the 

preferred embodiments, it is apparent that certain changes and modifications in 

20 addition to those mentioned above may be made from the basic features of this 

invention. In addition, there are many different types of computer software and 
hardware that may be utilized in practicing the invention, and the invention is not 
limited to the examples described above. Accordingly, it is the intention of the 
Applicants to protect all variations and modification within the valid scope of the 

25 present invention. It is intended that the invention be defined by the following 

claims, including all equivalents. 



